An Efficient Method for Large Margin Parameter Optimization in Structured Prediction Problems

We consider structured prediction problems with a parametrized linear prediction function, and the associated parameter optimization problems in large margin type of discriminative training. We propose a dual optimization approach which uses the restricted simplicial decomposition method to optimize a reparametrized dual problem. Our reparametrization reduces the dimension of the space of the dual function to one that is linear in the number of parameters and training examples, and hence independent of the dimensionality of the prediction outputs. This in conjunction with simplicial decomposition makes our approach efficient. We discuss the connections of our approach with related earlier works, and we show its advantages. Technical Report C-2007-87 Dept. Computer Science University of Helsinki ∗Huizhen Yu is with the Helsinki Institute for Information Technology (HIIT), University of Helsinki, Finland. †Juho Rousu is with the Department of Computer Science, University of Helsinki, Finland.